Visual Enhancement Capsule Network for Aspect-based Multimodal Sentiment Analysis

Multimodal sentiment analysis, which aims to recognize the emotions expressed in multimodal data, has attracted extensive attention both academia and industry. However, most of current studies on user-generated reviews classify overall sentiments hardly consider aspects user expression. In addition, social media are usually dominated by short texts expressing opinions, sometimes attached with images complement or enhance emotion. Based this observation, we propose a visual enhancement capsule network (VECapsNet) based fusion for task aspect-based analysis. Firstly, an adaptive mask memory is designed extract local clustering information from opinion text. Then, aspect-guided mechanism constructed obtain image related aspect phrases. Finally, module interactive learning presented classification, takes phrases as query vectors continuously capture features correlated affective entities multi-round iterative learning. Otherwise, due limited number review datasets at present, build large-scale dataset Chinese restaurant reviews, called MTCom. The experiments single-modal demonstrate that our model can better more applicable general than existing methods. experimental results verify effectiveness proposed VECapsNet.